Processing CCM Logs

I'm engaging in a project to move our Cisco Unified Call Manager (aka CUCM, here known as CCM) Call Detail Report (CDR) logs into an internally managed system. I've elected to use Splunk for this task because of its flexibility and power in processing log entries. �The goal is to make a Splunk app that is generic enough to be used by others, and provides a high level of visibility and reporting for our phone logs. The final goal, which may or may not be possible, is to also provide complete costing information, so finance can know (with a reasonable level of certainty) how much our phone bill will be, and who is causing the greatest number of utilization charges. Over the course of the next few blog posts, I'll go into the details of our implementation, problems solved, and reports made. But let's get started.

The first question is why -- why get logs out of CCM in the first place? The short answer to this question is that CCM's internal CDR management software is pretty limited. The scenario we were in was that our provider was giving us very little billing detail, and we had almost no visibility into our overall phone usage. We were also evaluating switching providers, and at the very least pursuing a significant reduction in service. We needed the visibility, and we didn't have it.

To implement this, I decided that we'd use Splunk to do all the reporting, searching, and etc. To get data into Splunk, we would use Cisco's built-in log SFTP, which drops one log file per X interval onto an SFTP server. The log format, though, is in CSV, and isn't particularly Splunk-compatible. To make the transition, I used a custom Perl script to read CCM format CSV files, and output Splunk-compatible event logs. By putting a Perl script between the CCM and Splunk, we can also enrich the logs with a variety of data -- see a future post for that.

Finally, I built the reporting. The goal was to have:

• a general dashboard with a variety of information
• a fraud dashboard with key toll fraud (or misuse) indicators
• an operational dashboard with key system health reporting
• a financial dashboard with system costs
• a User-At-A-Glance dashboard with a summary for any particular user of the system.

Additionally, for traditional CDR reporting, we wanted a guided search for all the calls involving a given local user, and a guied search for calls involving a given remote phone number.

So tune in to the upcoming posts for the details of how I implemented this, problems I encountered, and how you might use it in your environment.